Generalization of the Dynamic Ordering for the One-Sided Block Jacobi SVD Algorithm: II. Implementation

We have designed, implemented and tested (by simulation on a serial computer) the new dynamic ordering for the parallel one-sided block-Jacobi SVD algorithm. Our idea is based on the estimation of the cosines of principal angles between two block columns X and Y of the same width without explicitly forming the matrix product XY (or Y X) and computing its SVD. Instead, we propose to use a fixed number 2q of iterations in the Lanczos algorithm applied to the symmetric 2x2 block Jordan-Wielandt matrix with zero diagonal blocks, 21-block XY and 12-block Y X; the order of the Jordan-Wielandt matrix is the sum of the block column widths. However, the matrix blocks XY and Y X are never formed explicitly; the needed matrix-vector multiplications are computed exchanging intermediate product vectors between two processors that host the block column X and Y . After computing 2q iterations, the Frobenius norm of an auxiliary tridiagonal matrix of order 2q estimates the square root of twice the sum of squares of q largest cosines (representing q smallest principal angles) between X and Y . In the parallel algorithm using p processors, these weights can be used for choosing p pairs of block columns, which are far from orthogonality with respect to those q smallest angles. We show how to implement this new parallel ordering in the distributed paradigm of parallel computing using the Message Passing Interface (MPI). First numerical results obtained by simulation show that the one-sided parallel dynamic ordering can lead to a substantial decrease of the number of parallel iteration steps needed for the convergence as compared to a cyclic ordering.